Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add (some) google resource support #36

Merged
merged 5 commits into from
Nov 13, 2024
Merged

Add (some) google resource support #36

merged 5 commits into from
Nov 13, 2024

Conversation

fiskhest
Copy link
Contributor

@fiskhest fiskhest commented Nov 8, 2024 •
edited
Loading

I was mildly saddened when I realised this utility would not be able to help me perform a major migration within $COMPANY terraform states. With the help of a colleague, this PR contains support for generating the imports for resources we found, together with tests.

This is not a complete implementation supporting all google resources, but a starting point for anyone else that does google migrations, adding new ones is pretty straight forward.

Maybe adding a minor hint to the README about the coverage not being 100% would be nice, it took me some time to figure out why the imports this generated was total bologna. 🍝

  • make fmt
  • To run only the tests, run command make test
go test -race -v ./...
?       github.com/kishaningithub/tf-import-gen [no test files]
=== RUN   Test_ComputeTerraformImportForResource
=== RUN   Test_ComputeTerraformImportForResource/For_aws_iam_role_policy_attachment
=== RUN   Test_ComputeTerraformImportForResource/For_aws_lambda_permission
=== RUN   Test_ComputeTerraformImportForResource/For_aws_security_group_rule_with_source_security_group_id
=== RUN   Test_ComputeTerraformImportForResource/For_aws_security_group_rule_with_cidr_blocks
=== RUN   Test_ComputeTerraformImportForResource/For_aws_security_group_rule_with_prefix_list_ids
=== RUN   Test_ComputeTerraformImportForResource/For_aws_api_gateway_resource
=== RUN   Test_ComputeTerraformImportForResource/For_aws_api_gateway_deployment
=== RUN   Test_ComputeTerraformImportForResource/For_aws_api_gateway_stage
=== RUN   Test_ComputeTerraformImportForResource/For_aws_api_gateway_method_settings
=== RUN   Test_ComputeTerraformImportForResource/For_aws_cloudwatch_event_target
=== RUN   Test_ComputeTerraformImportForResource/For_aws_api_gateway_method
=== RUN   Test_ComputeTerraformImportForResource/For_aws_api_gateway_integration
=== RUN   Test_ComputeTerraformImportForResource/For_aws_route_table_association
=== RUN   Test_ComputeTerraformImportForResource/For_aws_iam_user_policy_attachment
=== RUN   Test_ComputeTerraformImportForResource/For_aws_emr_instance_group
=== RUN   Test_ComputeTerraformImportForResource/For_aws_backup_selection
=== RUN   Test_ComputeTerraformImportForResource/For_aws_vpc_endpoint_route_table_association
=== RUN   Test_ComputeTerraformImportForResource/For_aws_cognito_user_pool_client
=== RUN   Test_ComputeTerraformImportForResource/For_aws_ecs_cluster
=== RUN   Test_ComputeTerraformImportForResource/For_aws_ecs_task_definition
=== RUN   Test_ComputeTerraformImportForResource/For_aws_wafv2_web_acl
=== RUN   Test_ComputeTerraformImportForResource/For_aws_autoscaling_schedule
=== RUN   Test_ComputeTerraformImportForResource/For_aws_appautoscaling_target
=== RUN   Test_ComputeTerraformImportForResource/For_aws_appautoscaling_policy
=== RUN   Test_ComputeTerraformImportForResource/For_aws_ecs_service
=== RUN   Test_ComputeTerraformImportForResource/For_google_monitoring_alert_policy
=== RUN   Test_ComputeTerraformImportForResource/For_google_monitoring_notification_channel
=== RUN   Test_ComputeTerraformImportForResource/For_google_service_account_iam_binding
=== RUN   Test_ComputeTerraformImportForResource/For_google_service_account_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_privateca_ca_pool_iam_member.pool
=== RUN   Test_ComputeTerraformImportForResource/For_google_privateca_certificate_template_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_project_iam_binding
=== RUN   Test_ComputeTerraformImportForResource/For_google_project_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_sql_database_instance
=== RUN   Test_ComputeTerraformImportForResource/For_google_sql_user
=== RUN   Test_ComputeTerraformImportForResource/For_google_iap_tunnel_instance_iam_binding
=== RUN   Test_ComputeTerraformImportForResource/For_google_bigquery_dataset_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_bigquery_table_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_kms_crypto_key_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_kms_crypto_key_iam_binding
=== RUN   Test_ComputeTerraformImportForResource/For_google_cloud_run_service_iam_binding
=== RUN   Test_ComputeTerraformImportForResource/For_google_secret_manager_secret_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_secret_manager_secret_iam_binding
=== RUN   Test_ComputeTerraformImportForResource/For_google_storage_bucket_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_storage_bucket_iam_binding
=== RUN   Test_ComputeTerraformImportForResource/For_google_compute_subnetwork_iam_binding
=== RUN   Test_ComputeTerraformImportForResource/For_google_organization_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_project_iam_custom_role
=== RUN   Test_ComputeTerraformImportForResource/For_google_pubsub_topic_iam_binding
=== RUN   Test_ComputeTerraformImportForResource/For_google_pubsub_topic_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_pubsub_subscription_iam_member
=== RUN   Test_ComputeTerraformImportForResource/For_google_resource_manager_lien
=== RUN   Test_ComputeTerraformImportForResource/For_google_monitoring_uptime_check_config
=== RUN   Test_ComputeTerraformImportForResource/For_everything_else
--- PASS: Test_ComputeTerraformImportForResource (0.01s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_iam_role_policy_attachment (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_lambda_permission (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_security_group_rule_with_source_security_group_id (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_security_group_rule_with_cidr_blocks (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_security_group_rule_with_prefix_list_ids (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_api_gateway_resource (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_api_gateway_deployment (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_api_gateway_stage (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_api_gateway_method_settings (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_cloudwatch_event_target (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_api_gateway_method (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_api_gateway_integration (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_route_table_association (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_iam_user_policy_attachment (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_emr_instance_group (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_backup_selection (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_vpc_endpoint_route_table_association (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_cognito_user_pool_client (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_ecs_cluster (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_ecs_task_definition (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_wafv2_web_acl (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_autoscaling_schedule (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_appautoscaling_target (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_appautoscaling_policy (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_aws_ecs_service (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_monitoring_alert_policy (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_monitoring_notification_channel (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_service_account_iam_binding (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_service_account_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_privateca_ca_pool_iam_member.pool (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_privateca_certificate_template_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_project_iam_binding (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_project_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_sql_database_instance (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_sql_user (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_iap_tunnel_instance_iam_binding (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_bigquery_dataset_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_bigquery_table_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_kms_crypto_key_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_kms_crypto_key_iam_binding (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_cloud_run_service_iam_binding (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_secret_manager_secret_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_secret_manager_secret_iam_binding (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_storage_bucket_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_storage_bucket_iam_binding (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_compute_subnetwork_iam_binding (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_organization_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_project_iam_custom_role (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_pubsub_topic_iam_binding (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_pubsub_topic_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_pubsub_subscription_iam_member (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_resource_manager_lien (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_google_monitoring_uptime_check_config (0.00s)
    --- PASS: Test_ComputeTerraformImportForResource/For_everything_else (0.00s)
=== RUN   TestImport_ShouldSerializeAsTerraformImportStatements
--- PASS: TestImport_ShouldSerializeAsTerraformImportStatements (0.00s)
=== RUN   TestImports_ShouldSerializeAsMultipleTerraformImportStatements
--- PASS: TestImports_ShouldSerializeAsMultipleTerraformImportStatements (0.00s)
=== RUN   TestImport_ShouldGenerateHelpfulMessageWhenResourceDoesNotSupportImport
--- PASS: TestImport_ShouldGenerateHelpfulMessageWhenResourceDoesNotSupportImport (0.00s)
=== RUN   Test_GenerateImports_ShouldGenerateImportsForAllResourcesWhenNoFiltersAreGiven
=== RUN   Test_GenerateImports_ShouldGenerateImportsForAllResourcesWhenNoFiltersAreGiven/only_root_resources
=== RUN   Test_GenerateImports_ShouldGenerateImportsForAllResourcesWhenNoFiltersAreGiven/resources_in_child_module
=== RUN   Test_GenerateImports_ShouldGenerateImportsForAllResourcesWhenNoFiltersAreGiven/resources_in_deeply_nested_child_module
=== RUN   Test_GenerateImports_ShouldGenerateImportsForAllResourcesWhenNoFiltersAreGiven/google
--- PASS: Test_GenerateImports_ShouldGenerateImportsForAllResourcesWhenNoFiltersAreGiven (0.01s)
    --- PASS: Test_GenerateImports_ShouldGenerateImportsForAllResourcesWhenNoFiltersAreGiven/only_root_resources (0.00s)
    --- PASS: Test_GenerateImports_ShouldGenerateImportsForAllResourcesWhenNoFiltersAreGiven/resources_in_child_module (0.00s)
    --- PASS: Test_GenerateImports_ShouldGenerateImportsForAllResourcesWhenNoFiltersAreGiven/resources_in_deeply_nested_child_module (0.00s)
    --- PASS: Test_GenerateImports_ShouldGenerateImportsForAllResourcesWhenNoFiltersAreGiven/google (0.00s)
=== RUN   Test_GenerateImports_ShouldGenerateImportsForResourcesForGivenAddress
=== RUN   Test_GenerateImports_ShouldGenerateImportsForResourcesForGivenAddress/filtering_by_module
=== RUN   Test_GenerateImports_ShouldGenerateImportsForResourcesForGivenAddress/filtering_by_resource
--- PASS: Test_GenerateImports_ShouldGenerateImportsForResourcesForGivenAddress (0.00s)
    --- PASS: Test_GenerateImports_ShouldGenerateImportsForResourcesForGivenAddress/filtering_by_module (0.00s)
    --- PASS: Test_GenerateImports_ShouldGenerateImportsForResourcesForGivenAddress/filtering_by_resource (0.00s)
=== RUN   Test_GenerateImports_ShouldGenerateHelpfulCommentForResourceThatCannotBeImported
--- PASS: Test_GenerateImports_ShouldGenerateHelpfulCommentForResourceThatCannotBeImported (0.00s)
=== RUN   Test_GenerateImports_ShouldGenerateImportsForResourcesForMultipleAddresses
=== RUN   Test_GenerateImports_ShouldGenerateImportsForResourcesForMultipleAddresses/filtering_by_module
--- PASS: Test_GenerateImports_ShouldGenerateImportsForResourcesForMultipleAddresses (0.00s)
    --- PASS: Test_GenerateImports_ShouldGenerateImportsForResourcesForMultipleAddresses/filtering_by_module (0.00s)
=== RUN   Test_GenerateImports_ShouldNotGenerateDuplicateImportsForResourcesForMultipleAddresses
=== RUN   Test_GenerateImports_ShouldNotGenerateDuplicateImportsForResourcesForMultipleAddresses/filtering_by_module
--- PASS: Test_GenerateImports_ShouldNotGenerateDuplicateImportsForResourcesForMultipleAddresses (0.00s)
    --- PASS: Test_GenerateImports_ShouldNotGenerateDuplicateImportsForResourcesForMultipleAddresses/filtering_by_module (0.00s)
PASS
ok      github.com/kishaningithub/tf-import-gen/pkg     1.027s
=== RUN   TestTerraformStateJsonParserAddressComputationForRootResources
=== RUN   TestTerraformStateJsonParserAddressComputationForRootResources/root_resource_without_index
=== RUN   TestTerraformStateJsonParserAddressComputationForRootResources/root_resource_with_index_as_integer_and_included_in_address
=== RUN   TestTerraformStateJsonParserAddressComputationForRootResources/root_resource_with_index_as_integer_and_not_included_in_address
=== RUN   TestTerraformStateJsonParserAddressComputationForRootResources/root_resource_with_index_as_string_and_included_in_address
=== RUN   TestTerraformStateJsonParserAddressComputationForRootResources/root_resource_with_index_as_string_and_not_included_in_address
--- PASS: TestTerraformStateJsonParserAddressComputationForRootResources (0.00s)
    --- PASS: TestTerraformStateJsonParserAddressComputationForRootResources/root_resource_without_index (0.00s)
    --- PASS: TestTerraformStateJsonParserAddressComputationForRootResources/root_resource_with_index_as_integer_and_included_in_address (0.00s)
    --- PASS: TestTerraformStateJsonParserAddressComputationForRootResources/root_resource_with_index_as_integer_and_not_included_in_address (0.00s)
    --- PASS: TestTerraformStateJsonParserAddressComputationForRootResources/root_resource_with_index_as_string_and_included_in_address (0.00s)
    --- PASS: TestTerraformStateJsonParserAddressComputationForRootResources/root_resource_with_index_as_string_and_not_included_in_address (0.00s)
=== RUN   TestTerraformStateJsonParserDoesNotAddModuleNameTwice
=== RUN   TestTerraformStateJsonParserDoesNotAddModuleNameTwice/1_level_repeated_module_address
--- PASS: TestTerraformStateJsonParserDoesNotAddModuleNameTwice (0.00s)
    --- PASS: TestTerraformStateJsonParserDoesNotAddModuleNameTwice/1_level_repeated_module_address (0.00s)
=== RUN   TestTerraformStateJsonParserAddressComputationForNestedResources
=== RUN   TestTerraformStateJsonParserAddressComputationForNestedResources/nested_resource_without_index
=== RUN   TestTerraformStateJsonParserAddressComputationForNestedResources/nested_resource_with_index_as_integer_and_included_in_address
=== RUN   TestTerraformStateJsonParserAddressComputationForNestedResources/nested_resource_with_index_as_integer_and_not_included_in_address
=== RUN   TestTerraformStateJsonParserAddressComputationForNestedResources/nested_resource_with_index_as_string_and_included_in_address
=== RUN   TestTerraformStateJsonParserAddressComputationForNestedResources/nested_resource_with_index_as_string_and_not_included_in_address
--- PASS: TestTerraformStateJsonParserAddressComputationForNestedResources (0.00s)
    --- PASS: TestTerraformStateJsonParserAddressComputationForNestedResources/nested_resource_without_index (0.00s)
    --- PASS: TestTerraformStateJsonParserAddressComputationForNestedResources/nested_resource_with_index_as_integer_and_included_in_address (0.00s)
    --- PASS: TestTerraformStateJsonParserAddressComputationForNestedResources/nested_resource_with_index_as_integer_and_not_included_in_address (0.00s)
    --- PASS: TestTerraformStateJsonParserAddressComputationForNestedResources/nested_resource_with_index_as_string_and_included_in_address (0.00s)
    --- PASS: TestTerraformStateJsonParserAddressComputationForNestedResources/nested_resource_with_index_as_string_and_not_included_in_address (0.00s)
PASS
ok      github.com/kishaningithub/tf-import-gen/pkg/internal/parser     1.019s

@kishaningithub
Copy link
Owner

@fiskhest Thanks a lot for adding support for several of the Google resources.

Feel free to update the readme regarding the coverage.

I see some tests are failing when I triggered the pipeline can you look into those? Thanks again 😊

@fiskhest
Copy link
Contributor Author

Oops, should be sorted now. As for README, I gave it a shot; please leave a suggestion if you prefer some other way of expressing the fact.

kishanbsh-gilead
kishanbsh-gilead suggested changes Nov 12, 2024
View reviewed changes
pkg/convertor.go Outdated Show resolved Hide resolved
pkg/convertor.go Outdated Show resolved Hide resolved
pkg/convertor.go Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
kishaningithub
kishaningithub requested changes Nov 12, 2024
View reviewed changes
Copy link
Owner

@kishaningithub kishaningithub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some very minor knit picks. Post that this looks good to merge

pkg/convertor.go Outdated Show resolved Hide resolved
pkg/convertor.go Outdated Show resolved Hide resolved
@fiskhest
Copy link
Contributor Author

🙏🏼 should be the last of it. Thanks! 🎉

@kishaningithub
Copy link
Owner

Merging it! Thanks a lot for the contribution @fiskhest !

@kishaningithub kishaningithub merged commit 836cc2a into kishaningithub:main Nov 13, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kishanbsh-gilead kishanbsh-gilead kishanbsh-gilead requested changes

@kishaningithub kishaningithub kishaningithub approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@fiskhest @kishaningithub @kishanbsh-gilead